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Abstract 

We describe the supercuspidal representations of Sp4(i^), for F a non-archimedean local field 
of residual characteristic different from 2, and determine which are generic. 

Introduction 

Let F be a locally compact non-archimedean local field, with ring of integers o^, maximal ideal 
pi?, and residue field kp- Whereas every irreducible supercuspidal representation of GL7v(-^) is 
generic - i.e., has a Whittaker model - this is no longer true for classical groups over F. The 
existence of non-generic supercuspidal representations of classical groups is significant and bears 
many consequences; let us cite a few of them in local representation theory (global consequences are 
heavy as well, see for instance |12j): First, of course, the fact that the definition of the L-function 
attached to a representation of a classical group is available only for generic representations, thanks 
to the work of Shahidi; in particular, a characterization through local factors of the local Langlands 
correspondence for a classical group is not fully available. Also, reducibility of parabolic induction 
is completely understood for GLn{F) while in classical groups results are complete only in the case 
of generic inducing representations. 

The most celebrated example of a non-generic supercuspidal representation is the representation 
010 of Sp4(F), induced from the inflation to Sp4(0i7) of the cuspidal unipotent representation 
of Sp4(A;f) constructed by Srinivasan (se [12] again). To our knowledge, until recently, most known 
non-generic supercuspidal representations were level zero, in particular level zero representations 
induced from the inflation of a cuspidal unipotent representation of the reductive quotient of a 
maximal special parahoric subgroup. Our purpose in this paper is to exhaust the non-generic 
supercuspidal representations of Sp4(F) in odd residual characteristic. 

It is certainly no surprise, and part of the folklore in the subject, that level zero supercuspidal 
representations of Sp4(-F) coming from cuspidal representations of Sp4(fci?) are generic if and only if 
the corresponding cuspidal representation is, and that the only non-generic cuspidal representation 
of Sp4(A;i?) is Srinivasan's ^lo- For the sake of completeness we include proofs of these results. 
Actually we prove that generic level zero supercuspidal representations of Sp27v(-^) are obtained by 
inducing the inflation of a generic cuspidal representation of the reductive quotient of a maximal 
parahoric subgroup of Sp2jv(-^)! on the condition that this parahoric subgroup is special. 
For positive level supercuspidal representations, the situation is not as simple. Our point of view 
is to use the exhaustive construction given by the second author in [2T]: those representations are 
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induced from a set of types, generalizing Bushnell-Kutzko types for GLiy(F). We give necessary 
and sufficient conditions on those types for the induced representation to be generic. We obtain 
surprisingly many (at least with respect to our starting point) non-generic representations. 

Let us be more precise. Positive level supercuspidal representations of Sp4(F) fall into four cate- 
gories, according to the nature of the skew semi-simple stratum [A, n, 0, (3] at the bottom level of 
the construction ( §2.ip : 

(I) The first category starts with a skew simple stratum with corresponding field extension F[P] 
of dimension 4 over F. Here non-generic representations are obtained only when F[P]/F is 
the biquadratic extension, and when a binary condition involving (3 and the symplectic form 
is fulfilled. 

(II) The second category starts with a skew simple stratum with corresponding field extension 
F[P] of dimension 2 over F. Non-generic representations are obtained whenever the 
skew hermitian form attached to the symplectic form is anisotropic (or equivalently, when 
the quotient group J/J^ involved in the construction, which is a reductive group over kp, is 
anisotropic). 

(III) In the third category, the stratum is the orthogonal sum of two two-dimensional skew simple 
strata [Aj, rij, 0, /Jj], i = 1,2. Non-genericity occurs only when F[Pi] is isomorphic to F[P2], 
and again when a binary condition involving P1P2 and the symplectic form is fulfilled. 

(IV) All representations in the fourth category - when the stratum is the orthogonal sum of a skew 
simple stratum and a null stratum, both two-dimensional - are non-generic. 

The main character in the proof is indeed a would-be character: To a stratum as above is attached 
a function ip/^ on Sp4(-F) and the crucial question is whether there exists a maximal unipotent 
subgroup U of Sp4(F) on which ip/^ actually defines a character (see [5]); this question is easily 
solved in ^ where Proposition 13.41 lists the exact conditions alluded to above. Whenever there is 
no such subgroup U, we prove in ^that the corresponding representations are not generic, using 
a criterion of non-genericity given in ^1.21 (this says essentially that if a cuspidal representation 
c-Indj''''^^^A is generic, then there is a long root subgroup on which A is trivial). 

Now assume that there is a maximal unipotent subgroup U on which ipp is a character. A type 
attached to our stratum is a representation k ® a of a. compact open subgroup J, where k is 
a suitable /3-extension attached to the stratum and o" is a cuspidal representation of some finite 
reductive group J/J^ attached to the stratum (see ^2.ip . The fundamental step is Theorem 14.31 
stating that the representation k contains the character ipjs of J U. This implies genericity in 
cases (I) and (III), where a is just a character of an anisotropic group. But for cases (II) and (IV) (in 
which, we should add, ipis defines a degenerate character of U) we have to deal with the component 
a, with opposite effects. In case (II), the inflation of a contains the restriction to J D U of a 
character of U and genericity follows. On the contrary, in case (IV), the inflation of a does not 
contain the restriction to J nU of a character of U and the resulting cuspidal representation is not 
generic. 

We have several remarks to add concerning these results. First, genericity for positive level super- 
cuspidal representations of Sp4(-F) only depends on the stratum itself, on the symplectic form, and 
on the representation a (the "level part"). It does not depend on the choice of a semi-simple 
character attached to the stratum. 
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The second remark is that the proofs are quite technical, often on a case-by-case basis. The present 
work can of course be regarded as a first step towards understanding non-genericity in classical 
groups, but even the case of Sp27v(-^) niight not be just an easy generalisation. In particular, 
the precise conditions for genericity are surprisingly complicated but it seems to us that they 
do not admit a simple unified description, as for level zero representations. For example, non- 
generic positive level supercuspidal representations can be induced from either special or non-special 
maximal compact subgroups of Sp4(-F), and likewise for generic representations. 

Finally, we have deliberately stuck to the construction of supercuspidal representations of Sp4(F) 
via types. Another very fruitful point of view uses Howe's correspondence. Indeed, in a very 
recent work ([H]), Gan and Takeda study the Langlands correspondence for GSp4(-F) and obtain 
in the process a classification of non-generic supercuspidal representations of GSp4(-F) in terms of 
Howe's correspondence; they also announce a sequel dealing with Sp4(-F). A dictionary between 
these two points of view would of course be very interesting, especially if it can provide some insight 
about the way non-generic supercuspidal representations of Sp4(-F) fit into L-packets. For example, 
in case (I), the genericity of the supercuspidal representation depends on the embedding of /? in 
the symplectic Lie algebra: up to the adjoint action of Sp4(F), there are two such embeddings, 
precisely one of which gives rise to non-generic representations. This suggests that representations 
in these two sets might be paired to form L-packets. A closer investigation of such phenomena 
would certainly deserve some effort. 

Notation 

Let F be a locally compact non-archimedean local field, with ring of integers Op, maximal ideal 
pF, residue field kp and odd residual characteristic p = char A; j?. On occasion wp will denote a 
uniformizing element of F. Similar notations will be used for field extensions of F. We let I'p 
denote the additive valuation of F, normalised so that vf{F^) = Z. We fix, once and for all, an 
additive character ipp oi F with conductor pp. 

Let V be a 2A^-dimensional i^-vector space, equipped with a nondegenerate alternating form h. 
By a symplectic basis for V, we mean an ordered basis {e_Ar, e_i, ei, eAr} such that, for 
l<i,j<N, 

h{ei,ej) = h{e-i,e-.j) = 0, h{e^i,ej) = 5ij, 
where 5ij is the Kronecker delta. 

Let A = Endi7'(y) and let ~ denote the adjoint anti-involution on A associated to /i, so 

h{av,w) = h{v,aw), for a & A, v,w G V. 

We will let G be the corresponding symplectic group G = {g G GLp(y)/'g = g"^}, or G = Sp2jv(-^) 
whenever a symplectic basis is fixed. For most of the paper N will be 2. Similarly G will denote 
either GLp{V) or GL2jv(i^). 

Skew semi-simple strata in A are the basic objects in what follows. We recall briefly the essential 
notations attached to those objects and refer to [20] for definitions. 

Let [A, n, 0, f3] be a skew semisimple stratum in A. Then A is a self-dual lattice sequence in V and 
defines a decreasing filtration {ai(A),i € Z} of vl by OiT'-lattices aj(A) = {x € A/\/k G Z, xA{k) C 
A{k + i)}. We put a(A) = Oo(A), a self-dual o^-order in A. We will also need P(A) = a(A)^ n G 
and -Pi (A) = (1 + ai(A)) n G for i > 1. Note that the lattice sequence A gives rise to a valuation i>\ 
on V (j6.ip and the filtration {aj(A), z € Z} gives rise to a valuation on A, also denoted by 
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Next, /3 is a skew element in A: (5 = — /3, and n is a positive integer with n = —v\{(3). Furthermore 
the algebra E = F[(3] is a sum of fields E = ®\^iEi, corresponding to a decomposition V =-L'=i 
of V as an orthogonal direct sum, and accordingly of A: A = E'^j^A' with A*(A:) = A(A;) fl V^, and 
of /?: (5 = Ti\^-^f3i. The centralizer i? of /? in ^ is -B = ®\^iBi where Bi is the centralizer of j3i in 
EndF • 

Last, for j3 a skew element of A we define ijjj^ as the following function on G: i^ipix) = if) (tr(/?(x — 1))), 



1 Generalities on genericity 
1.1 Genericity 

The results of this subsection are valid in a much more general setting than the remainder of this 
paper so we temporarily suspend our usual notations. 

Let G = G{F) be the group of F-rational points of a connected reductive algebraic group G defined 
over F. Let S be a maximal F-split torus in G with G-centralizer T, let B be an F-parabolic 
subgroup of G with Levi component T, and let U be the unipotent radical of B. 

Let X be a smooth (unitary) character oiU = U(F). The torus S = ^{F) acts on the set of such 
characters by conjugation. We say that x is nondegenerate if its stabilizer for this action is just 
the centre Z of G. 

Example 1.1. Let G = Sp4(F), which we write as a group of matrices with respect to some 
symplectic basis, let T be the torus of diagonal matrices, and let U be the subgroup of upper 
triangular unipotent matrices in G. Any character x of [/ is given by 



X 



/lux y \ 

1 V X 

1 -It 

Vo 1 y 



ippio-u + bv), 



for some a,b G F, and it is easy to see that x is nondegenerate if and only if a, b are both non- 
zero. Moreover, there are four orbits of nondegenerate characters of U, given by the class of b in 

FX/(FX)2. 

Returning to a general connected reductive group G, let vr be a smooth irreducible representation 
of G. We say that vr is generic if there exist U = U(F) as above and a nondegenerate character x 
of U such that 

HomG(vr,Indgx) y^O. 

Note that, since all such subgroups U are conjugate in G we may choose to fix one. Moreover, we 
need only consider nondegenerate characters x up to T-conjugacy. A basic result here is: 

Theorem 1.2 ([15] Theorem 3). Assume G is split over F. Let tt be a smooth irreducible repre- 
sentation of G and let x be a nondegenerate character of a maximal unipotent subgroup U of G. 
Then 

dimcHomG(7r,Ind{^x) ^ 1- 

On the other hand, and again in a general G, when dealing with supercuspidal representations we 
may not bother about the nondegeneracy of the character in the definition of genericity, a fact that 
will be useful in the sequel. Indeed: 
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Lemma 1.3. Assume G is split over F. Let tt be a smooth irreducible supercuspidal representation 
of G. Let JJ be a maximal connected unipotent subgroup of G and let x be a character ofU = U(F) 
such that 

HomG(^,Indgx) /O. 
Then the character x is nondegenerate. 

Proof. Assume x is degenerate and use the definition of nondegeneracy in [7], 1.2, as well as the 
corresponding notation: there is a simple root a such that the restriction of x to is trivial. 
The character x is then trivial on the subgroup {U[a)^ U^er) of U . 

We claim that this subgroup contains the unipotent radical of a proper parabolic subgroup P 
of G. The assumption Hom ^(Tr, Ind^x) 7^ provides us, by Frobenius reciprocity, with a non-zero 
linear form A on the space V oi tt which satisfies A o 7r(n) = x('u)A for any u U. Since A is in 
particular A^-invariant, we get that the space Vn of A^-coinvariants is non-zero, which contradicts 
cuspidality. 

We now prove the claim. Let A be the set of simple roots as in From 21.11, the unipotent 
radical of the standard F-parabolic subgroup P/ of G attached to the subset / = A — {a} is U,i,(/) 
where ^{I) is the set of positive roots that can be written a + P with /? either or a positive root. 
Hence the elements in ^{I) other than a are positive roots 7 of length at least 2, and U^(/) is 
directly spanned by U(„) and the U(^) for non-divisible such roots 7. From [6j, Theorem 4.1, for 
any such 7 we have U(^)(F) C C/der (recall the characteristic of F is not 2), hence A^ = U^(/)(F) 
is contained in (f/(Q,), f/der) as asserted. ■ 

Now let TT be an irreducible supercuspidal representation of G. We suppose, as is the case for 
all known supercuspidal representations, that vr is irreducibly compact-induced from some open 
compact mod centre subgroup of G. Then (the proof of) [5j 1.6 Proposition immediately gives us 
the following: 

Proposition 1.4. Let K be an open compact mod centre subgroup of G and p an irreducible 
representation of K such that vr = c-lnd^p is an irreducible, so supercuspidal, representation of G. 
Then tt is generic if and only if there exist a maximal connected unipotent subgroup JJ of G and a 
character X of U = U(i^) such that p\K OU contains x\^ f^U. Moreover, if this is the case then 
Hom (^(tt, Ind[/x) 7^ so x nondegenerate, and the character x\K D U occurs in p\K U with 
multiplicity 1. 

1.2 A criterion for non-genericity 

Now we look more closely at the situation for symplectic groups so we return to our usual notation: 
G = Sp2n{F). In particular, using Proposition 11.41 and a decomposition of G, we will obtain a 
criterion to determine when an irreducible representation of G is not generic. 

Let T denote the standard (diagonal) maximal split torus of G, let U be the subgroup of all upper 
triangular unipotent matrices in G, and put B = TU, the Borel subgroup of all upper triangular 
matrices in G. Let $ = $(G,T) be the root system and, for 7 E let denote the corresponding 
root subgroup. Let W denote the Weyl group Ng{T)/T; by abuse of notation, we will also use W 
for a set of representatives in the compact maximal subgroup Kq = Sp2Ar(Oi7') of G. 

We write Ki for the pro-unipotent radical of Kq, so that Kq/Ki ~ Sp27v(^F). We note that 
B n Kq/B n Ki is the standard Borel subgroup (of upper triangular matrices) of Kq/Ki, that 
T n Kq/T n Ki is the diagonal torus, and that W is the Weyl group. 
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Let Ii denote the inverse image of the maximal unipotent subgroup U nKo/U CiKi of Kq/Ki, that 
is, the pro-unipotent radical of the standard Iwahori subgroup / consisting of matrices which are 
upper triangular modulo pi?. Then the Bruhat decomposition for Kq/Ki gives 

Kq/Ki = {Br\Ko)Wh/Ki. 

Since i^i C /i, we obtain Kq = {Br\ Kq)WIi. Finally, using the Iwasawa decomposition G = BKq 
(since Kq is a good maximal compact subgroup of G), we obtain 

G = BWh. (1.5) 

Now we can use this decomposition to translate Proposition 11.41 into a sufficient condition for 
non-genericity of compactly-induced supercuspidal representations of G. 

Proposition 1.6. Let J he a compact open subgroup of G and A an irreducible representation of J 
such that TT = c-IndjA is an irreducible supercuspidal representation of G. Let U be the subgroup of 
all upper triangular unipotent matrices in G and we also use the other notations from above. Then 
TT is generic if and only if there exist w G W, p £ Li and a character x of U such that contains 
the character of U'^ . In particular, 

if TT is generic then there are p € /i and a long root 7 E ^> such that contains the 
trivial character of n fXy. 

We remark that, in our symplectic basis, the long roots correspond to the entries on the anti- 
diagonal. 

Proof. Since all maximal unipotent subgroups of G are conjugate to U, Proposition 11.41 implies 
that TT is generic if and only if there exist g £ G and a character x of C/ such that A contains the 
character of J n [/^. Now we use the decomposition (II. Sh to write g = bwp, with b (z B, w W 
and p G Ii. Since = U and x'^ is another character of U, we can absorb the b and the result 
follows on conjugating by p. 

The final assertion follows since the derived subgroup f/^^ contains for some long root 7. ■ 

2 The supercuspidal representations of Sp4(F) 

What we seek is a complete list of which supercuspidal representations of Sp4(F) are generic, which 
are not. So we start with a description of positive level supercuspidal representations of Sp4(F) 
- level zero supercuspidal representations are obtained by inducing from a maximal parahoric 
subgroup V the inflation of a cuspidal representation of the (finite) quotient of "P by its pro-p-radical. 
We are then in a position to state our main theorem, identifying non-generic representations from 
our list. We end the section with a proof of the theorem for level zero representations. The proof 
for positive level occupies the remaining sections. 

2.1 The positive level supercuspidal representations 

In this section we describe the construction of the positive level supercuspidal representations of 
Sp4(F). We refer to [21] for more details and for proofs of the results stated here. 
The construction begins with a skew semisimple stratum [A, n, 0, /3] in A such that a(A) fi i3 is a 
maximal self-dual order normalized by in B. There are essentially four cases here. In the first 
two, the stratum is actually simple: 
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(I) "maximal case": [A,n, 0,/3] is a skew simple stratum and E = F[/3] is an extension of F of 
degree 4. 

(II) "2 then case": [A, n, 0, (3] is a skew simple stratmn, E = F[(3] is an extension of F of degree 
2, and ao(A) is maximal amongst (self-dual) Oi?-orders in A normalized by E^ . 

Otherwise, we have a splitting V = -LV^ oi [A, n, 0, f3] into two 2-dimensional F-vector spaces, 
and we write: A* for the lattice sequence in given by A*(A;) = A{k) nV^, for k £ Z; /3j = VpV, 
where 1* is the projection onto with kernel V^^^; if /3i 7^ then nj = otherwise rii = 0. 

Then n = max {ni, 722}. 

(III) "2+2 case": for i = 1,2, [A*, nj, 0, Pi] is a skew simple stratum and Ei = F[f3i] is an extension 
of F of degree 2. 

(IV) "2+0 case": [A^, ni, 0, is a skew simple stratum and Ei = F[l3i] is an extension of F of 
degree 2; /?2 = 0, so that in V'^ we have the null stratum [A^, 0,0,0], and Oo(A'^) is maximal 
amongst (self-dual) OiT'-orders in = Endi7'(y^). 

We often think of pvp as a degenerate case of piip by thinking of a null stratum as a degenerate 
simple stratum. 

In each case, we have the subgroups = H^{(3,A), .P = J^(/?, A) and J = J(/3, A) of G. We 
write = fl G, and similarly for the other groups. There is a family C(/3, A) of rather 
special characters of H^, called semisimple characters; one of their properties is the fact that their 
restriction to Pi{A) for suitable i is equal to ipjs. For each 6 G C(/?, A), there is a unique irreducible 
representation r] of containing 6. In each case, there is a "suitable" extension k of r/ to a 
representation of J, which we call a (3-extension ~ see below for more details of this step. 

The extensions E, Ei in each case come equipped with a non-trivial galois involution, which we 
write ~ as usual. We use the same notation for the induced involution on the residue fields kE, ki] 
note that this involution may be trivial. Then the quotient J/J^ has one of the following forms: 

(I) k]^ = {x £ kE ■ XX = 1}; 

(II) U{l,l){kE /kp) or k^ x k^ li E/F is unramified; SL2{kF) or 02{kF) if E/F is ramified; 

(III) kl X fcl; 

(IV) kl X SL2{kF). 

Let a be the inflation to J of an irreducible cuspidal representation of J/J^. (Note that, in the 
case of 02(^f) in (flB . this just means any irreducible representation of the (anisotropic) dihedral 
group 02(^^)0 

Now we put A = K (g) cr and vr = c-Indj A is an irreducible supercuspidal representation of G. All 
irreducible supercuspidal representations of G can be constructed in this way (though we remark 
that often we cannot, as yet, tell when two such representations are equivalent). 

Finally in this subsection, we recall briefly some properties of the /^-extensions which we will need, 
especially in the cases dll]) and pvp where their construction is somewhat more involved. Indeed, 
in cases (jl]) and OIB . J/J^ has no unipotent elements so there is never any problem here. 

We define another skew semisimple stratum [Am, rim, 0, (5] as follows: 
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• in cases (jTj) and piip . we have = A, Um = n; 

• in case (jll]), A^ is a self-dual 0£;-lattice sequence in V with ao{Am) C ao(A) minimal amongst 
(self-dual) OiT'-orders normalized by , and = —i^AmiP)^ 

• in case OVj) . we take A^ a self-dual OiT'-lattice sequence in V with ao(A^) C ao(A^) minimal 
amongst (self-dual) OiT'-orders, A^ = A^ _L A^, and rim = — 2^Am(/3)- 

In each case, we have the subgroups = H^{l3,Am), Jm = J^{P,Am) of G, and we put = 
Hm n G etc. Let 0™. £ C(/3, A^,) be the transfer of 0, that is 6*^ = ta,a™,,/3^ in the notation of [20] 
§3.6, and let ??m be the unique irreducible representation of containing 9m- We form the group 



Then (see [2lj) there is a unique irreducible representation of which extends rj and such that 
f] and 7/m induce equivalent irreducible representations of Pi(Am). Moreover, if Ig{f]) denotes the 
^(-intertwining space of ry, we have 



A /3-extension of rj is an irreducible representation k of J such that k\jx = fj. 
2.2 The main theorem 

Theorem 2.1. The non-generic supercuspidal representations of Sp4(-F) are the following. 

(i) The positive level supercuspidal representations attached to a skew semisimple stratum 
[A, n, 0, /9] as above and such that: 



• either [A, n, 0,/?] is a sum of non-zero simple strata (cases (0), W\) and and there 
is no maximal unipotent subgroup of G on which ifjf^ is a character; 

• or [A, n, 0, /?] is the sum of a non-zero simple stratum and a null stratum in dimension 



(ii) The level zero supercuspidal representations induced from the inflation to a maximal parahoric 
subgroup V of a cuspidal representation a of V/V^ , where is the pro-p-radical of V, 
satisfying one of the following: 

(a) V is attached to a non-connected subset of the extended Dynkin diagram of G, that is, 
V /V^ is isomorphic to Sp{2, kp) x Sp{2, kp). 

(b) V is isomorphic to Sp4(0i?) and a is a non-regular cuspidal representation of Sp4{kp). 

For positive level representations the theorem will follow from Proposition 13.41 which establishes the 
conditions on (3 for there to exist a maximal unipotent subgroup of G on which -0^ is a character, 
and from Theorem 14.51 and section [5l The proof for level zero representations is given below, with 
a more detailed list. 



ji = {Pi{Am)nB)j\ 




2 (case im). 
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2.3 The generic level zero representations of Spa^rC^) 

Note that for finite reductive groups, the notion equivalent to genericity is cahed regularity: a 
representation of Sp2jv(fcF) is called regular if it contains a nondegenerate character of a maximal 
unipotent subgroup. Part ([n]) of the above theorem, i.e. the level zero case, actually holds for 
Sp27v (F) 5 as a consequence of Propositions 11.41 and 11.61 

Proposition 2.2. Let V be a maximal parahoric subgroup of Sp2^(-F) with pro-p-radical and 
let a be a cuspidal representation of V/V^. The representation vr of Sp2i\f{F) induced from the 
inflation of a to V is irreducible supercuspidal. It is generic if and only if the quotient V jV^ is 
isomorphic to Sp2jv(^F) o,nd a identifies to a regular cuspidal representation of Sp2jv(^F)- 

Proof. Up to conjugacy, we may assume that V is standard; in particular, using the notation in 
11.21 V contains /. Then our standard V is the group of invertible and symplectic elements in the 
order 

/ Miiop) Mi^2N-i{0F) Mi{)p-p}) \ 
21 = M2N- j,i(pF) M2N-2i{0F) M2N- i,i{oF) for some integer i, < i < iV. 

V Mi(Pi.) Mi^2N-i{pF) Mi{OF) J 

Assume first that V /V^ is isomorphic to Sp27v(^f)) i-e. i = or A^, and use Proposition [T31 If f is 
regular, then vr is generic. Conversely if vr is generic, there is a maximal unipotent subgroup U' and 
a character x' of U' such that a contains X|-pn(7'- The subgroup U' is conjugate to the subgroup 
U of all upper triangular unipotent matrices so, using the Iwasawa decomposition G = VB as in 
11.21 we may replace U' by U. Since a is cuspidal. Lemma [T31 applied to Sp27v(^f)5 tells us that 
X\vr\U identifies with a nondegenerate character of P fl C/ /"P^ n J7, a maximal unipotent subgroup 
of Sp2jv(^F)) hence a is regular. 

Assume now that 1 < i < A^ — 1: then V jV^ is isomorphic to Sp2j(fc_F) x Sp2Ar_2j(^-F)) the relevant 

/* *\ 

entries being those in I * I in the above description of 21. Assume for a contradiction that 

\* */ 

TT is generic: from Proposition 11.61 plus the inclusion I\ C P, there exist w and a character 
X of [/ such that a contains the character y^^ oi V r\\J'" . Let IT"' = P n [/"'/P^ n and let Y" 
be the character of defined by x""- The group U"^ is a maximal unipotent subgroup of V jV^ 
(a simple combinatoric argument suffices here). We will show that, since x is trivial on f/der; the 
character x"' is degenerate, thus contradicting the cuspidality of o. 

To fix ideas, suppose that w = \. The intersection of the image of \J with Sp2j(/c_F) is the subgroup 
IJi of upper triangular unipotent matrices while the image of U^er contains the simple long root: 
the restriction of x to f/j is degenerate hence a cannot be cuspidal (Lemma ll.Sp . 

In general, observe that w must map the A positive long roots (corresponding to the antidiagonal 
entries in [/) onto a set £ oi N long roots that correspond to the long root entries in U'^. Those A 
long roots separate into i long roots in U'^ n Sp2i{kF) and N — i long roots in fl Sp27v-2i(^-F)- 
The A^ — 1 positive not simple long roots corresponding to antidiagonal entries in i/jer are sent 
onto a subset of A^ — 1 long roots in f : only one is missing, so either in R Sp2j(A;i?) or in 
tj"^ n Sp27v-2i(^-F)i the unique long root entry that does not belong to the derived group does 
belong to U^^/. on this group, the restriction of is degenerate, so a is not cuspidal. ■ 
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2.4 The cuspidal representations of Sp4(Fq) 

We come back to Sp(4). It has been known for a long time that among the cuspidal representations 
of Sp4(A;i?), only one is non-regular, the famous representation of Srinivasan ([l7] II. 8. 3, [E]); it 
is the unique cuspidal unipotent representation of Sp^{kF). Yet this common knowledge lacks of a 
reference in the modern setting of Deligne-Lusztig characters, we thus pause here to detail the list 
of cuspidal representations of Sp(4,Fq) as they arise from the Lusztig classification. The necessary 
background and notations are taken from the book [10] , specially chapter 14. 

For this subsection only we let G = Sp(4,Fg) and we let F be the standard Frobenius on G, acting 
as X I— > x'^ on each entry, so that G^ = Sp(4,Fq). We let G* be the dual group SO{5,¥q) with 
standard Frobenius F* . 

Deligne-Lusztig characters of G^ are parameterized by pairs (T*, s), T* an F*-stable maximal torus 
of G* and s an element of T*^ , up to G*^ -conjugacy. A rational series of irreducible characters of 
G^ is made of all irreducible components of Deligne-Lusztig characters i?j?*(s) where the rational 
conjugacy class of s (i.e. the G*^ -conjugacy class of s) is fixed. Rational series of characters 
are disjoint and exhaust irreducible characters of G^ . Cuspidal (irreducible) characters are those 
characters that appear in some B^f* (s) for a minisotropic torus T* and do not appear in any 
where the torus T* is contained in a proper i^*-stable Levi subgroup of G* . 

Let s be a rational semi-simple element contained in an F*-stable maximal torus T* of G*, let 
Cg*{s) (resp. C^*(s)) be its centralizer in G* (resp. the connected component of its centralizer) 
and let W{s) (resp. W°{s)) be the Weyl group of Gg*{s) (resp. Cg, (s)) relative to T*, contained 
in the Weyl group W{T*) of G* relative to T*. 

For w in W{T*), there exists an F*-stable maximal torus of G* of type w with respect to T* 
and containing s if and only if w belongs to W''{s). Letting x be the type of T* with respect to 
some split torus, one defines by the formula 

w&W°{s) 

a proper character x(s) which is a multiplicity one sum of regular irreducible characters, each 
appearing with multiplicity ±1 in Deligne-Lusztig characters underlying the series attached to s. 
We have 

{R^,{s),I^,{s))^^ = C^.dW{sr^* and {x{s),x{s)) = \{W{s) /W^is))^* \ 

and the results in loc. cit., §14, imply the following, for the rational series of characters attached 
to s: 

(i) If only minisotropic rational maximal tori contain s, all characters in the series are cuspidal. 
The number of regular cuspidal characters in the series is the number of components of x(s)- 

(ii) If no minisotropic rational maximal torus contains s, there is no cuspidal character in the 
series. 

(iii) If at least one minisotropic rational maximal torus and at least one non-minisotropic rational 
maximal torus contain s, no cuspidal character in the series (if any) is regular. 

(iv) If exactly one minisotropic rational maximal torus (up to rational conjugacy) and at least 
one non-minisotropic rational maximal torus contain s, there is no cuspidal character in the 
series. 
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The Weyl group of G* has 8 elements. A rational maximal torus of type w with respect to a 
split torus is minisotropic if and only if w is either a Coxeter element h (there are two of them, 
conjugate in the Weyl group) or the element of maximal length wq. Rational points of such a torus 
are conjugate to Tq^^ , isomorphic to 

IK4 = ker A'px for w = h, 

]fC2 X IfC2 = ker A'px ^jpx x ker A'j^x yj^x for w = wq. 

The table below lists the families of geometric conjugacy classes of rational semi-simple elements 
of G* through a representative sq (not necessarily rational) in the diagonal torus 




using the following notation: we fix Q^, a primitive {q^ — l)-th root of unity in , and we let 

a,2 = cf-\ C2 = Cf+\ (2,1 = Cr' andC = C|+'. 



In cases 13 and 14, x{^) = R^*{s) is irreducible, cuspidal and regular (it actually contains any 
nondegenerate character of a maximal unipotent subgroup) we 

8 4 

equivalence classes of such representations. 

Cases 4, 5, 6, 7, 8 and 9 give no cuspidal representations ([n]), neither do cases 10 and 12 (jiv]). Cases 
2 and 3 each determine two rational series, in which again (jivl) applies: they contain no cuspidal. 

Missing cuspidals ([IT] II. 8. 3, [E]) now must come from cases 1 and 11. Indeed case 11 produces 
two rational series, one of which satisfying (jn]), but the other for a torus of type wq. Here 
x{s) = Rj.*{s) is the sum of two irreducible, cuspidal and regular (but for different choices of a 

nondegenerate character of a maximal unipotent subgroup) representations and we get 2 — ^ 
equivalence classes of such representations. 

Last, case 1 gives the so-called unipotent series, which for Sp(4, F^) contains exactly one cuspidal 
representation ([13] 5 Theorem 3.22), non-regular by (|in|) . 
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Case 


So 


condition 


number 


W [So) 


\VV[So)/W {So)\ 


1 


r(i,i) 




1 


W 


1 


2 


**(-!, -1) 




1 


{sa',Sa'+2P') 


2 


3 


t*{-i,i) 




1 




2 


4 


t*{c,i) 




2 




1 


5 


t*(C,-i) 




q-3 
2 


1 


2 


D 




A* ^ 4-1 


9-3 
2 


{Sa') 


1 
i 


1 




C^±i 

/"?' ^ 4-1 


(g-3)((/-5) 
8 


1 
i 


1 
i 


8 






(g-l)2 
4 


1 


1 


9 




>7 / 1 -1 


(g-l)(g-3) 
4 


1 


1 


10 




a,i ^ ±1 


(9-1) 
2 




1 


11 




d,i 7^ ±1 


(9 - 1) 
2 


1 


2 


12 






19 - 
2 




1 


13 


**(C2,l! (2,1) 


Cii 7^ ±1 

C^l 7^ C2,l 


(9-l)(9-3) 
8 


1 


1 


14 


^*iCl,2' (1,2) 


Cl,2 / ±1 


g2-l 

4 


1 


1 
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3 The function ?/?^ on maximal unipotent subgroups 



A key step in the determination of Whittaker functions in GLAr(F) in [5] is the construction of a 
maximal unipotent subgroup U of GL]\f{F) on which ipp defines a character {loc. cit. propositions 
2.1 and 2.2). This wih be a key step indeed in the determination of generic supercuspidal repre- 
sentations of Sp4(-F): the existence of such a subgroup on which defines a non degenerate 
character wiU turn out to be a sufficient condition for genericity (see whereas the non existence 
wiU imply non genericity (see fj5]). In cases where such a U exists but ■0/3 is degenerate, we will find 
both generic and non generic representations. 

3.1 The quadratic form h{v,f3v) 

Proposition 3.1. Let (3 he an element of A such that (3 = —(3 and let tpp he the function on G 
defined hy ip/3{x) = ip {iT{f3{x — 1))), x G G. The following are equivalent: 

(i) There exists a maximal unipotent subgroup U of G such that the restriction of ipf^ to U is a 
character of U. 

(ii) There exists a maximal unipotent suhgroup U of G such that 0/3 (x) = 1 for all x G U^er- 

(iii) There exists a totally isotropic flag of suhspaces of V : 

{0} c c ^2 c V3 c y 

such that f3Vi C Vi^i for i = 1,2. 

(iv) The quadratic form v 1— > h{v, (3v) on V has non trivial isotropic vectors. 

Proof. The equivalence of the first three statements is straightforward and a variant of [5j 2.1; 
note that a maximal flag of subspaces of V determines a maximal unipotent subgroup of G if and 
only if it is totally isotropic. 

Certainly (iii) implies (iv): a basis vector v for Vi satisfies h{v, f3v) = since V2 is its own orthogonal. 
Assuming (iv), let u be a non-zero vector in V such that h{v,(3v) = and put Vi = Span {v}, 
V3 = V^. If Pv and v are colinear, let V2 be any totally isotropic 2-dimensional subspace of V 
containing Vi, otherwise put V2 = Span {v,(3v}: (iii) is satisfied since, for a totally isotropic flag 
as in (iii), the conditions (3Vi C V2 and (3V2 C V3 are equivalent (recall (3 = —(3). ■ 

Remark 3.2. Assume the conditions in Proposition \ 3.1\ hold and let U he a maximal unipotent 
suhgroup of G attached to a totally isotropic flag {0} C 14 C V2 C V3 C F .such that f3Vi C Fj+i for 
i = 1,2. A simple inspection shows that the character 0^ of U is non degenerate if and only if (3Vi 
is not contained in Vi and (3V2 is not contained in ¥2- 

We need to investigate those cases where the element (3 appears in a skew semi-simple stratum 
[A, n, 0,/3] as listed in ^2.1\ We need an extra piece of notation in cases HI or HIl where the stratum 
is simple: the field extension E = F[(3] has degree 4 or 2; we let Eq be the field of fixed points 
of the involution x ^ x on E, so that [E : Eq\ = 2, and we define a skew-hermitian form SonV 
relative to E/Eq by 

/i(af , li;) = tr£;/p(a(5(t>, li;)) ioi all a & E,v,w G V. (3-3) 

(This notation will also be used in case IIIII when Ei and E2 are isomorphic, with E = Ei.) The 
determinant of 6 belongs to i^^ if : = 2; it is a skew element in E^ if [E : F] = 4:. 
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Proposition 3.4. Let (3 he an element of A appearing in a skew semi-simple stratum [A, n, 0, /3] 
as in ' i2.1\ The only cases in which there does not exist a maximal unipotent subgroup U of G on 
which ij)f^ is a character are the following: 

(i) The element (3 generates a biquadratic extension E = F[0\ of F (case \^ and the coset 
/3det (5) Ne/Eo[F'^) F^ is the N ^ / Eo{F^)-coset that does not contain the kernel of tr^;^/^. 

(ii) The element (3 generates a quadratic extension F = F[(3] of F ( case\l^ and the skew-hermitian 
form 6 on V, dim eV = 2, is anisotropic - that is, det (6) ^ N£;/p{F^). 

(iii) The symplectic space V decomposes asV = V^l.V'^ and the element jS decomposes accordingly 
as (3 = [5i + j32 where for i = 1,2, (3i generates a quadratic extension Fi = F[j3i] ( case I //-/] ). 
Fi is isomorphic to F2 and (3i/(32 ^ dei{5) Neip{F^). 

Remark 3.5. Let (3 be as above. Assume there exists a maximal unipotent subgroup U of G on 
which tpp is a character. Then: 

• in cases\^ and \II]\ the character tjjp of U is non degenerate; 

• in cages [HI and \IV\ the character ijjp of U is degenerate. 

The proof of those statements occupies the next two subsections. We recall that, up to isomorphism, 
there is exactly one anisotropic quadratic form on V: its determinant is a square and its Hasse- 
Minkowski symbol is equal to —{—1,—1)f ([H]) §63C). 

3.2 The biquadratic extension 

Let us examine the case of a maximal simple stratum (case H]) . The determinant of the quadratic 
form V I— > h{v,(3v) on V is the determinant of j3, i.e. Np/p{P). 

Lemma 3.6. The norm Np/p{P) of P is a square in F^ if and only if F is biquadratic. If this 
holds we have: Ne/e^{F'^) = F'^F^^. 

Proof. A four-dimensional extension of F is called biquadratic if it is Galois with Galois group 
Z/2Z X Z/2Z. Biquadratic extensions of F are all isomorphic, their norm subgroup is i^^^ (class 
field theory). The "if" part is then clear. Now assume that Np/p{P) is a square. Since /3 is skew 
and generates a degree four field extension of F, its square generates Fq over F and is not a 
square in F^ , while Np^ip{[3'^) = Np/p{l3) must be a square in F^ . We proceed according to 
ramification. 

If Fq is ramified over F: 0^ must have even valuation, its squareroot generates an unramified 
extension of Fq. So F/Fq is unramified and Np/p^{F^) is made of even valuation elements, i.e. is 

equal to F'^F^^ since 0^^ = o^{l + ppj. It follows that NpjpiF'^) = F^l 

If £'0 is unramified over F, we have in the residual field kp^: 

u S k^^ is a square in k^^^ if and only if Nf.^^ij.p[u) is a square in kp. 

We write = w-^pU with u G 0^^. Then Np^ip{0^) = zu'^J Npg/p{u). It follows that u must be a 
square and must have odd valuation: its square root generates a ramified extension of Fq. Then 
F is the extension -Eolo;] where is a uniformizing element in F, and Np/p^{F^) = (— a^)^Og^ = 

F^'F^^ because k^ C k^'^. It follows that Np/piF"") = ■ 
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If Ne/p{(3) is not a square, we are done. Assume from now on that E is biquadratic and use 
the form 6 defined in 13. 3[ Since (3 is skew and 5 is skew-hermitian, the element (35{v,v) belongs 
to Eq. Since V is one-dimensional over E the form 5{v,v) is anisotropic and the subset D{V) = 
{I35{v^v) I f G y, f 7^ 0} of Eq is one of the two cosets of N^i^^^E'^) in Eq . On the other hand, 

we have N^/Eoi^^) = F^Eq hence the set of non-zero elements in Ker tiEo/F is fully contained 
in one of those two cosets. Proposition 13.41 follows in this case (and Remark 13.51 directly follows 
from 13.21 since f3 generates a degree 4 extension of F). 

Remark. We can be more precise about this condition: if £"0 is unramified over F and 1^^?! = 3 [4], 
then h{(3v, v) is anisotropic if and only if f36{v, v) ^ F^ Eq ; otherwise h{l3v, v) is anisotropic if and 
only if I35(v,v) E F'^Ef. 

3.3 Cases II, III and IV 

The case numbered IIVI in ^2.11 is obvious. The set of isotropic vectors for the quadratic form 
h{v,(5v) is the subspace V"^. The flags that satisfy ISlT iii) are the flags that can be written in the 
form {0} C Fe-2 C Fe_2 + Fe-i C Fe_2 + Fe-i + Fei C V where {e_j, Cj} is a symplectic basis 
of V' for i = 1,2. 

Case HI] is also quite clear: as in case H] the element (36{v,v) belongs to £"0 = F^ hence h{Pv,v) = 
2f55{v, v) has isotropic vectors if and only if 5{v^ v) does. Furthermore a flag {0} C Vi C V2 C V3 C 
V as in l3.1l must have the form Vi = Fv, where v is non zero and isotropic for 6, and V2 =< v, (3v >. 
Since belongs to F^ we always have f3V2 = V2 so, if defines a character of the corresponding 
unipotent subgroup of G, this character is degenerate. 

We finish with case IIIII We have Y = y^_Ly^ and /5 = /3i + /32- For f E y, writing v = V2 
on y = V^J-V'^, we get h{v,f3v) = h{vi,(3ivi) + h{v2, P2V2)- The determinant of this form is the 
product Ne^/p{(3i)Ne^/p{I32)- For the form to be anisotropic, the determinant must be a square 
hence 

NE,/Fif^i) = NE^/piP2) mod 

Each Pi is skew with characteristic polynomial — {—Np-/p{(3i)): the class of —Np.jp{f3i) mod 
the squares determines, up to isomorphism, the extension Ei. So if Ei and E2 are not isomorphic 
we are done. 

We pursue assuming they are and let E = Ei ~ £2- We may see V as a vector space over E and 
define 6 as in 13.31 The decomposition V = V^l.V'^ is orthogonal for 6 as well, and for i = 1,2, 
Vi I— > h{vi,f3iVi) is an anisotropic quadratic form on V^: a (non zero) isotropic vector for h{v,(3v) 
must have the form v = + v'^ with E V^, 7^ 0. We then have: 

h{v,l3v) = hiv\l3iv^) + h{v'^,f32V^) = 2(3i5{v\v^) + 2l32 5iv\v^), 

and 13.41 follows. 

The remark on the non degeneracy of '0/3 on U whenever it defines a character follows from the fact 
that and V"^ are anisotropic for h{v,f3v): the corresponding totally isotropic flag has the form 
(with notations as above) {0} C Vi =< + v"^ >C V2 =< v'^ + v'^,f3iv^ + f32v'^ >C V-^^ C V. We 
never have PV2 ^ V2 unless = /?| - but this belongs to case HIl not to case IIIII 
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4 Generic representations 



In section[3l we have found necessary and sufficient conditions for there to exist a maximal unipotent 
subgroup [/ of G on which defines a character. When there is such a U, this gives us a candidate 
for trying to build a Whittaker model, as is the case in GLn (F) (see [5] ) . In this section we consider 
the case where there is such a U. 

4.1 Characters and /3-extensions 

Proposition 4.1 (cf. [5j Lemma 2.10]). Let [A, n, 0,/5] be a skew semisimple stratum in A. Let 
6 € C(A, 0,/3) be a skew semisimple character and let U be a maximal unipotent subgroup of G such 
that i^isluder = 1- ^^en 

Unfortunately, we have been unable to find a unified proof of this Proposition; the proof is therefore 
rather ugly, on a case-by-case basis, and we postpone it to the appendix. 

We continue with the notation of Proposition 14.11 Then we can define a character of = 
(J n U)H^ by 

Q^{uh) = ijp{u)e{h), for n G J n [/, h e . 
Notice that this is a character, since J normalizes and intertwines 9 with itself. 

Corollary 4.2 (cf. [3 Lemma 2.11]). Let rj be the unique irreducible representation of which 
contains 9. Then the restriction of r] to nU contains the character ippljinjj- 

Proof. The proof is essentially identical to that of ^ Lemma 2.11]. We recall that \i0{x,y) = 
9[x,y] defines a nondegenerate alternating form on the finite group J^/H^ ( |20l Proposition 3.28]). 
Notice also that the image of H in J^/H^ is a totally isotropic subspace for the form kg, 
since 9 extends to a character 0^ of H^. Now we can construct rj by first extending to (the 
inverse image in of) a maximal totally isotropic subspace of /H^ and then inducing to J^. In 
particular, r] contains and hence V'/3|jinc/- ^ 

Now put = {JnU)J^ = (P(A)nS^n[/)J^ Note that, if J/ is anisotropic then J n?7 = J^nC/ 
so Ji = Ji. 

Theorem 4.3 (cf. [l<j\ Theorem 2.6]). Let [A, n, 0, /3] be a skew semisimple stratum in A as listed 
in ^2.1\ and let k be a (5-extension ofrj to J as described there. Assume there is a maximal unipotent 
subgroup U of G such that V'/jIr/^er ~ '^^^ ^■^^ notation above. Then: 

In particular, we deduce that the restriction of k to J fl {/ contains the character ip/3\jnu (cf- [3 
Lemma 2.12]). Moreover, except in case IIV|, and case HIl when J/J^ is isotropic, this means that 
the restriction of the simple type A to J n ?7 contains ipfsljnu^ since A = k. 

Proof. The proof is in essence the same as that of [16] Theorem 2.6], which it may be useful to 
read first: because of the similarities, we do not give all the details here. 

We begin by proving 

^^'^{j^nU)m®p-V- (4.4) 
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We prove this first in cases HI and IIIII Here = {x £ E/xx = 1} is a maximal torus of G. Then 
J = E^J^ and vr = c-IndjK. Since J nU = nU, Corollary 14.21 imphes that k contains tp/sljnu- 
Hence vr contains V/? and, since ipp is then a nondegenerate character, n contains ip/sljnu with 
multiplicity one (Proposition II. 4|) . Hence &f3\(jinu)H'^ occurs in -q with multiplicity precisely one 
and (14. 4p follows (see [ISl Lemma 2.5]). Note that this already gives the Theorem cases HI and IHI^ 
since E^ is maximal and = in these cases. 

Now we consider the other cases HIl and IIVI Recall that U is given by a flag {0} C Vi C V2 C V3 C ^ 
(see Proposition 13. ip . described in §3.31 What we need here is to define a parabolic subgroup Pq 
of G, with unipotent radical Uq contained in U and with a specific Levi factor Mq conforming to A 
in the sense of [U §10]. We achieve this according to the case as follows. 

From ^3.31 the unipotent subgroup U is attached to a flag 

{0} C Fw-i C Fw-i + FPw-i C Fw-i + F(3w-i + FI3wi C V 

where {w-i^wi} is a Witt basis for V over E. We then let Mq be the stabilizer of the 
decomposition V = Ew-i © Ewi and Pq be the stabilizer of the flag {0} C Ew-i C V. 

(HVT) Here U is attached to a flag {0} C Fe_2 C Fe_2 + Fe_i C Fe_2 + V^cV. We can 
pick 62 so that {6-2,62} is a symplectic basis of adapted to A^. We then let Mq be the 
stabilizer of the decomposition V = Fe-2 © -F62 and Pq be the stabilizer of the flag 

{0} C F6_2 C F6_2 dV. 

Let kg be the form defined in the proof of Corollary 14.21 In each case, we have the following 
properties (see [U §10] and |2H Lemma 5.6, Corollary 5.10]): 

(i) [/, and have Iwahori decompositions with respect to (Mo,-Po)j 

(ii) J"*^ n Uq/H^ n f/o is a totally isotropic subspace of / with respect to the form kg; 

(iii) n Mq n U/H^ n Mo n [/ is a maximal totally isotropic subspace of n Mq/H^ n Mq; 

(iv) there is an orthogonal sum decomposition 

_ J^HMq f J^HUq ^ n \ 

m ~ mnMo \Wr]Tk ^ IpTvu^ J ' 

where C/g~ is the unipotent subgroup opposite to Uq relative to Mq. 

Then (J^ fl U)H^ has an Iwahori decomposition with respect to {Mq,Pq) and (J^ n U)H^/H^ ~ 
n Mo n U/H^ n Mo n [/ _L n C/o/i?^ n C/o is a maximal totally isotropic subspace of J^/H^. 
In particular, from the construction of Heisenberg extensions, equation (j4.4p follows. 

For the final stage, as in the construction of /3-extensions, there is an 0£;-lattice sequence Am such 
that = (P^(Am)ni?) J^, and P^(A'") still has an Iwahori decomposition with respect to (Mo, Pq). 
Defining fj to be Ind^^B^, we see that ?7|J^ = 77, and one checks that 

Ind^^'^^-^^T? ~ Ind'^l^^-'^Vm. 

Since this uniquely determines f/, from the definition of /3-extension we get that k| ji = 77, and the 
result follows. ■ 
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4.2 The positive level generic supercuspidal representations of Sp4(-F) 



Here we show that ah the positive level supercuspidal representations which are not in the list of 
Theorem 12.11 are indeed generic. 

Theorem 4.5. Let vr = c-Indj A he a positive level irreducible supercuspidal representation of 
G = Sp4(F) with underlying skew semisimple stratum [A, n, 0,/3]. Suppose that this stratum is not 
in cage 1/ VI and that there exists a maximal unipotent subgroup U of G such that ipp\uder ~ Then 
vr is generic. 

Proof. Except in case |IT] when J/J^ is isotropic, this is immediate from Theorem 14.31 since 
A|ji = n\ji in these cases so X\jr\U contains the character V'/^lint/; hence, by Proposition \1A\ vr is 
generic. 

Suppose now we are in case HIl and choose a Witt basis {w-i,wi} for V over E attached to the 
unipotent subgroup U as in the proof of Theorem l4.3l The quotient J/J^ ~ P(A)n-B^/Pi(A)n-B^ 
is then isomorphic to SL{2,kF) if E/F is ramified, to U{1, \){kE /kp) if E/F is unramified. 
A x\ 

{( Q 1 ) ^ '^ith respect to the £'-basis {uj^i,wi} and 



We have U r\ 



unB' 



{u{x) 



/I N{(3)x\ 

1 X 

1 

VO 1 / 



/x G F} 



13 wi wi 

— \ of V over r . 
2 ^ 



in the symplectic basis {w^i, Pw^^^, ^^-^^ 2 

Now recall that A = k,® a, for a some irreducible cuspidal representation of J/ J^. Note that all 
cuspidal representations a of ?7(1, l)(A;£;//ci?) or SL{2, kp) are generic, since the maximal unipotent 
subgroup is abelian and a cannot contain the trivial character, by Lemma 11.31 Here we only need 
the fact that (the inflation of) a restricted to J nU Ci B^ contains some character oi J H U Ci B^ , 

(I a h c\ 
1 X y 
1 z 
VO 1/ 

a character of C/, we see that a\J CiU contains x| J fl C/. Since K\jnu contains -0/3 (Theorem 14. 3p . we 
deduce that A| jnc/ contains the restriction to JnU of the character xV'/3 of U and, by Proposition ll.4| 
IT = is generic (and the character xi^fs is non-degenerate). ■ 



which must have the form u{x) 1— > 'tpp{ax) for some a in F. Since x ■ 



■ipp{ax) is 



4.3 Case |IV] 

In case IIV|, any maximal unipotent subgroup U of G on which ■ipi3 defines a character can be 

written as upper triangular unipotent matrices in a symplectic basis {e_2, e_i, ei, 62} as in the 

beginning of §3.31 Here {e_2, 62} is a symplectic basis of V'^ hence U PI B^ is made of matrices of 
/I x\ 

the form I 0010 I • Even though a, as a cuspidal hence generic representation of SL{2, kp), does 
Vo 1 / 

contain a non trivial character of J OU D B^ , this character is by no means the restriction of a 
character of U. This is an heuristic explanation of the fact that none of the case IIVI supercuspidal 
representations are generic, unfortunately not a proof. In this section, we prove a crucial result 
towards non-genericity. The last step will be taken in section 15. 4[ 
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Proposition 4.6. Let it = c-Indj A be a positive level supercuspidal representation from case \IV\ 
and let U be a maximal unipotent subgroup of G such that ip/sluder ~ ^- Then the restriction of X 
to J nU n is a sum of non-trivial characters. 

Proof. We retain all the notation of section SH] and write U as in the beginning of this section. We 
also write ^7^ = UnB"" = UnSpiV"^). Since J = (JnP(A)nS^) we have JnC/ = {JnU'^){J^nU) 
hence: ji = ( J n U'^)J^ and = {J n U'^){J^ n U)H^. We claim that 

(i) ji n = ifi n u"^; 

(ii) Ind;^\e/3 ~ ljnc/2 (g) r/. 

The first claim comes from the definitions in [T^ on p. 131. Indeed PI Sp(l^^) = n Sp(y^) = 
Pi(A^). For the second, we note first that ljnu"2 ® r] does define a representation of since 
J nU'^ normalizes {J^,rj) and r]\J^ n C/^ = rj\H^ fl is a multiple of ^pf}, trivial on f/^ {(32 = 0). 
Frobenius reciprocity gives a non-zero intertwining operator between those representations, which 
are irreducible (recall from §4.11 that Ind^jB^I = ri). 

We have A = K^a, with a an irreducible cuspidal representation of J/J^ = SL2{kp). In particular, 
a is generic and its restriction to /J^ is a sum of nondegenerate characters x- On the other hand, 
the second claim above, plus Theorem 14.31 tell us that k is trivial on J n C/^, q.e.d. ■ 



5 Non-generic representations 

Our goal in this section is to show that the positive level supercuspidal representations which are 
in the list of Theorem 12.11 are indeed non generic. For cases (U), ([Tl]) and piip we will prove that 
whenever there is no maximal unipotent subgroup on which the function ipjs is a character (see 
Proposition 13. 4|) . an irreducible supercuspidal representation of G = Sp4(F) with underlying skew 
semisimple stratum [A, n, 0, (3] is not generic. The main tool here is the criterion given in Proposition 
11.61 hence we will work out in some detail the restriction of tpj^ to one-parameter subgroups. This 
technique will also provide us with a last piece of argument to settle non-genericity in case pvp . 

5.1 The function on some one-parameter subgroups 

In a given symplectic basis (e_2, e_i, ei, 62) of V, we will denote by Uk, for k £ {—2, —1, 1, 2}, the 
following root subgroup : 



Uk = {I + X tk ] X G F} with tfc(ej 



■J J 



e_fc if j = k; 
otherwise. 



It is attached to a long root and has a filtration indexed by s G Z: Uk{s) = {1 + x t^ : x G Pp}- 

Let P be an element of A and let ipp be the function on G defined by Tpp{x) = ip{tT:l3{x — 1)), x G G. 
For k in {—2, —1, 1, 2}, let e{k) be the sign of k. 

Lemma 5.1. Fix g G G and k in {—2, —1, 1, 2}. Let s G Z. 

(i) For any X in F we have: ip/^ {1 + x^t^) = {e{k) x h{ge-k, f3ge-.k))- 

(ii) The character tpp of^U^ is non trivial on ^Uk{s) if and only if s < —vph{ge-k^Pg&-k)- 
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Proof. We keep the usual notation = gug , = g ug. We have for x F: 

V'/j (1 + X Hk) = V' (tr(/3x Hk)) = V' (tr(x/3^ tk)) = ^{x{ps)k,-k) 
while the {k, -k) entry of (33 is i(3'^)k,-k = e(/c)/i(e_fc, /3^e_fc) = e{k)h{ge_k, Pge-k)- ■ 

5.2 Proof for a maximal simple stratum 

We work in this paragraph with a maximal simple stratum [A, n, 0,/3] in FjudpY, satisfying the 
assumption in Theorem 12.11 namely, in terms of the form 5 (see l\3.3\i and ^3.2p : 

(i) E = F[(3] is a biquadratic extension of F with fixed points Eq under x i— > x; in particular 
Ne/e^{E-) = F-E^\ 

(ii) The subset D{V) = {l36{v,v) : v eV,v ^0} of Eq is the NE/Eo{E'')-coset in Eq that does 
not contain the kernel oftiEo/F- 

Note that lattice duality with respect to the form 6, defined by L"^ = {v ^ V : 6{v,L) '^Pe} for L 
an 0£;-lattice in V, coincides with lattice duality with respect to h. We may and do assume that 
the self-dual lattice chain A satisfies A(i)'^ = A(d — i), i G Z, with d = or 1. We also may and do 
assume that A is strict, i.e. has period 2. 

Lemma 5.2. Pick vq in V such that A{i) = p^fo for i € Z. We have VE^ivo, vq) = 1 — d and, for 
V (zV and s in Z; 

UFh{v,pv) = J UEo{P^{v,v)y, 
e[Eo/l< ) 

vfKv, fiv) = s w G A(s + i(n + d - 1)) - A{s + ^{n + d+ 1)). 

Proof. We have h{v,Pv) = —2tiCEo/FiP^{v,v)) ( §3.2p : the first statement is thus a consequence 
of the following property: 

Let x be an element of the F^E^'^-coset in Eq that does not contain the kernel of 

^"^Eo/F- Then VFi^Eo/FX= e{El/F) ^EoX. 

Indeed we have vf^'^Eo/fx = (.{eI/f) [^EqX + i^Eq (l + f)] where x i-^ x is the Galois conjugation 

X y 

of Eq over F. Let x and y in Eq such that — and — belong to —1 + pEa l^t u = x/y. Then 

X y 

u/u belongs to 1 + p^;;,, which implies that u belongs to F^Eq^ (easy checking according to the 
ramification of Eq over F). Hence x and y are in the same F^ EQ^-coset, that must be the coset 
containing the kernel of tr^;^/^, q.e.d. 

The second statement is now immediate. We can write vGVasv = uvo with u E. Then: 

VFh{v,f3v) = . [ve{I36{vq,vq)) + ve{uu)] 

e[Eo/i' ) e[E/Eo) 

hence i'Fh{v, [3v) = \{—n + 1 — d) + veu-, q.e.d. ■ 
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We now fix a symplectic basis (e_2, e_i, ei, 62) adapted to A: there are non-decreasing functions 
: Z — > Z such that 

A(J)= Pp'^'^es (jGZ). (5.3) 

se{-2, -1,1,2} 



Proposition 5.4. For any g € -P(A), for any k € {—2,-1,1,2}, the character ipp of ^Uk is non 
trivial on ^Uk^ P|-nj^-|^(A). 

Proof. Note first that i-*(A) normahzes -P^nj_,_]^(A), so ^Uk H P^nj^-^(A) is equal to ^Uk{s), where 
s G Z is defined by Uk n Pjnj^-^(A) = Uk{s). Using Lemma [612] for our lattice sequence A of period 
2 we get, for x (z F: 

l + xtke ^[|]+i(A) ^ Vi, xtkA{j) c A(j + [f ] + 1) 

^ Vi, e_k G A(j - 2afc(j) + [|] + 1 - 2i/i.x) 

^ i^A (e-k) > max {j - 2ak{j);j G Z} + [§] + 1 - 21/^^ 

<^ i/A (e-fe) > z^A (cfc) +[§]+!- "^t^FX 

^ 2uFX > 2uA (ck) + [f ] + 2 - d. 

1 -d + r 



Hence s = v\ (e^) + 1 + 
Since (7 belongs to -P(A) we have by Lemmas 15.21 and 16.2 



; we have to prove s < —VFh{ge-kT Pg^-k) (Lemma 15.1 



VFh{ge-k,l3ge-k) = vrKc-k, fie.-k) = -I'A (cfc) + d - 1 - ^{n + d- 1)). 



The condition we need is thus 



side is an integer by Lemma |5.2|) . 



< \{n — d —\) which holds for n > 1 (the right hand 



We now derive Theorem 12.11 in this case: the representation vr is not generic. Indeed if it was, we 
could find, by Proposition 11.61 some g G i-*(A) and some k G {-2,-1,1,2} such that A contains 
the trivial character of ^Uk H J. In particular the restriction of A to -Pjnj_^]^(A) would contain the 

trivial character of ^Uk H Prnj_|^-^(A). Since this restriction is a multiple of "0/3 this is impossible. 



5.3 Proof for a maximal semi-simple or non-maximal simple stratum 

We will in this section treat simultaneously cases (jll]) and OIip in Theorem 12.11 granted that: 

Lemma 5.5. Under the assumption of case 177)) in Theorem \2.1\. any splitting V = V^l.V'^ of V 
into two one- dimensional E -vector spaces splits the lattice chain A, that is, for any t G Z.- 

A{t) = A^{t)±A'^{t) with A\t)=A{t)nV\i = l,2. 

Proof. We know from section [3] that the given assumption amounts to the fact that the quadratic 
form V H-> h{v, (5v) on V has no non trivial isotropic vectors, which implies that the anti-hermitian 
form 5 onV defined by h{av,w) = tr£;/^(a(5(v, u;)) for all a £ E, v,w £ V, is anisotropic. From 
[T]A.2, the lattice chain underlying A is thus the unique self-dual 0£;-lattice chain in V. On the 
other hand self-dual 0£;-lattice chains A* in V^, i = 1,2, can be summed into a self-dual 0£;-lattice 
chain in V. Unicity implies that A is obtained in this way. ■ 
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We can now let [A,n,0,/3], with A = A^-LA^, (3 = (3i + and n = max {ni, 712}, be a maximal 
skew semi-simple stratum or a non-maximal skew simple stratum in Endi^F: we place ourselves 
in the situation of case ([Tl]) or case (jllip in Proposition 13.4^ case (jll]) being obtained by letting 
El = E2 = E and Pi = P2 = P- In particular Ei = F[Pi] is always isomorphic to E2 = F[P2] and 
one checks easily that the conditions (ii) or (iii) in Proposition 13.41 are equivalent to saying that the 
symplectic form satisfies: 

€ V\ yv2 € -h{vi,Pvi)h{v2,Pv2) i Ne^if{EI). 



For an homogeneous treatment regardless of the ramification over F of the quadratic extensions 
involved, we use the conventions explained in section [621 namely the lattices sequences A^, A^ and 
A are all normalized in such a way that they have period 4 over F and duality given by d = 1. 

Lemma 5.6. Let vi G , V2 G and v = vi + V2 ^ V . Let e = e{Ei/F). 

(1) UFh{v,Pv) =m:m.{uFh{vi,PiVi), vfK'"2,P2V2)}; 

(2) vt^,{v) >2vFh{v, jjv) — vaa-iLl , j. 

e e 

Proof. (pP) We have h{v,Pv) = h{vi,PiVi) + h{v2,P2V2)- The assertion follows if the valuations 
of h{vi,PiVi), i = 1,2, are distinct. If they are equal and finite, write h{v,Pv) = h{vi, PiVi){l + 

be congruent to —1 mod pi? (its opposite would be a square 

hence a norm) the result follows. 

([2]) Prom Lemma and (1) we get = min j=i^2Z^Ai(^j) = 'ciiva.i=i^2{2i'Fh{vi, PiVi) Ei^^ 

whence the result. ■ 



We fix symplectic bases {cj, e_i} of , i = 1,2, adapted to Aj and use notation 15. 3[ We denote by 
1^ and 1^ the orthogonal projections of V onto and V"^ respectively. 

We need information on the intersection of one-parameter subgroups ^Uk with subgroups of G of 
the following form (with ai,a2 positive integers and a = max {01,02}): 

L = (1 + Oa,(Ai) + a,,(A2) + a„(A)) n G. 

Lemma 5.7. Let g G -P(A); let gi = z^a (l* {ge—k)) G Z U +00. Then 

9TT r,T 9TT I \ [ l^A (Cfc) + maX i=i,2{aj - fiTj } 3 

^Uk n L = '^Uk{S) with s= 

Proof. We have for x G F and g G G (see P 2.9): 

1 + xHk e L x3tkA{j) C A^(j + oi) + + 02) for any j G 1. 

Since tkA{j) = p^^^'^ e-k we have, if g belongs to -P(A): 

1 + xHk^L ^ Vj, xge_k e + ai- Aak{j)) + A^{j + 02- 4afc(j)) 
for i = 1, 2, Vj, AuFX + gi>j - 4afc(i) + ai. 

We conclude with Lemma [ 
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We now apply this lemma to the subgroup L obtained with ai = + 1, i = 1, 2. Define integers li 
and I2 by li = i'ph{V (ge^k) 1 Al* (ge^k))- From Lemmas 15.11 and \5.7\ the character ipp of gU^g'^^ 
is non trivial on gUug^^ fl L if and only if 



i^A (cfc) + maxj=i,2{[^] + 1 - 5*} + 3 



4 



< -min{/i, ^2}- 



On the other hand we have gi = 2li by Lemma 16.41 and Lemma |6 . 31 relates rij = —i^AiiPi) 

and VEif^i] in any case, one checks: 

maxj=i^2{ + 1 — Si} = e — 2min {?!, ^2} with e = or 1. 

It follows that ■0/3 is non trivial on gU^g^^ H L if and only if i^a (cfc) < — 2min {/i, I2} — s, that is, 
z^A (cfc) < —"^i^phige-k, Pge-k) — e. Now Lemma EH gives us: 

z^A (5fe__fc) > 2uFh{ge-k,f3ge-k) - max{— ^i^, ^^^^^^ }. 

e e 

Since (7 belongs to -P(A), we have i^a (g^-k) = {^~k) = —^A (cfe) (Lemma 16. 2p . which implies the 
desired inequality. We have just proven: 

Proposition 5.8. For any g G -P(A), for any k € {—2,-1,1,2}, the character ipp of gUug^^ is 
non trivial on gU^g^^ H L, where 



L=[I + a[i^]+i(Ai) + a[:^]+,(A2) + a[n]+i(A)j n G. 

At this point we can derive Theorem 12.11 in cases and (jllB as in the previous subsection: the 
representation vr is not generic. Indeed if it was, we could find, by Proposition 11.61 some g G A) 
and some k £ {—2, —1, 1, 2} such that A contains the trivial character of ^Uk H J. In particular the 
restriction of A to the above subgroup L would contain the trivial character of ^Uk H L. Since this 
restriction is a multiple of V'/j this is impossible. 



5.4 Non-genericity in the degenerate case (IIVI) 

We let again [A, n,0, with A = A-^ © A^, P = Pi + P2 and n = max {ni, 712}, be a skew semi- 
simple stratum in Endj^y, but we assume that one of the two simple strata involved is null (case 
lIVp . Although there does exist a maximal unipotent subgroup on which 0^ is a character, this 
character is then degenerate (Proposition l3.4l and Remark l3.5p . We will show that the corresponding 
supercuspidal representation is non-generic, using Proposition 14.61 and the criterion 11.61 

Criterion 11.61 involves conjugacy by elements in an Iwahori subgroup. We will find convenient 
to use the standard Iwahori subgroup, and to use an Iwahori subgroup normalizing the lattice 
chain A. These conditions can both be fulfilled at the possible cost of exchanging A^ and A-^, 
that is, we have to complicate notations and let {1,2} = {r,s} with [A^, n^, 0, not null and 
[A*, n-s, 0, /3s] = [A**, 0,0,0]; in particular n = n^. 

Since the proof below is rather technical, we first sketch it, assuming P2 = 0. In a symplectic basis 
as in H'6.'6\ ipp does define a character of the upper and lower triangular unipotent subgroups, trivial 
on the long root subgroups U±2 corresponding to the null stratum but non trivial on the other long 
root subgroups U±i. As in the previous case, we have a subgroup L of J on which A restricts to a 
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multiple of ■0/3- We will show that for any Iwahori conjugate ^U±i, ijjj^ is non trivial on ^U±i R L. 
Next we will identify subsets X2 and X_2 of / such that, for g E X±2, ipp is again non trivial on 
^U±2 n L. The last step will be to show that for g £ I — X±2, if the representation A contains the 
trivial character of ^U±2 fl J, then it contains the trivial character of U±2 D J - this last possibility 
being excluded by Proposition 14.61 Hence, by Proposition 11.6^ the representation induced from A 
cannot be generic. 

We pick a symplectic basis (e_2, e_i, ei, 62) of V adapted to A and such that i-*(A) contains the 
standard Iwahori subgroup / consisting of matrices with entries m Op which are upper triangular 
modulo pF- We normalize lattice sequences Ai and A2 such that they have period 4 over F and 
duality invariant 1 and define e = if A^ contains a self-dual lattice, e = 1 otherwise ( ^6.2p . Let: 



L = + ai(A^) + 0[„j^-^(A'') + 0[n]_^^(A)) n G. 



This is a subgroup of (from the definition [19j p. 131) on which 9 restricts to ipf^ ([20] Lemma 
3.15). 

Lemma 5.9. Let k in {—2, —1, 1, 2}. For y E ^'(A), define yr = i^A(l'~(ye_fc)), ys = z^A(l*(ye_fc)) 
and Ir = VFh{V (yc-k) ,/3rl^ {y^~k))- The character ipp of yUky~^ is non trivial on yUky^^ if 
and only if either V{ye-k) =0 or V{ye-k) 7^ and 

{yr< yde-k) + [f ] + 1 - 2e 
\ys> -fkie-k) + 4/^ + 1 

Proof. We have h{v,(3v) = h{vr,(3rVr) (for v = vi + V2, Vi E V^), so with Lemmas [67 

uphiv, (3v)=t ^ V{v) E A'-(2t + [§] + 1 - e) - A'^(2t + [§] + 1 - e + 1 

In particular, if l''(ye_fc) = 0, the character ipp is trivial on yU^y^^ (Lemma 15. 1|) . 
We now assume V[ye-k) 7^ and get from Lemma l6.4t 



and 



2lr + 



n 
L2 



+ l-e. 



(5.10) 



We apply Lemma [5771 to L: 

yUky-^ n L = yUk{t)y-^ with t 



z^A(efc) + max{l - t/s, e - 2 Z^} + 3 



Using Lemma [5. II we conclude that the character ipp of yU^y ^ is non trivial on yll^y ^ fl -L if and 
only if t < —Ir whence the result (note that Vh{ek) = —v\{e-k) by Lemma [6. 2p . ■ 



Lemma 5.11. Let y e L. If \k\ = r, or if \k\ = s and V{ye^k) i ^''{i^Aie-k) + [H^]), the 
character -0/3 of yUky~^ is non trivial on yUky~^ H L. 



Proof. Since y belongs to P(A) we certainly have V{ye^k) £ {i^Ai^-k))^ l*(ye_fc) E h.^{i'\{e^k))i 
and: 

either (a) r(ye_fc) i A^u^ie^k) + 1) 

or (b) Viye^k) G A''{Me-k) + 1) and I'iye^k) ^ A'{i^A{e^k) + !)• 



(5.12) 
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Assume first (a). Then the first condition in Lemma 15.91 is satisfied and the second will hold if 
—i^Ai^-k) + 4/r + 1 < i^A(e-fc). But we have y,. = 2 + [^] + 1 — e (j5.10p whence 

Air = 2(i^A(e_fc) - [-J - 1 + e) < 2z.A(e-fc) - 1. 

Hence: 

if y € P{-^) a-nd V{ye^k) ^ {i^k{(^-k) + 1); "0/3 is non trivial on yUkV^^ H L. (5.13) 

The discussion now will rely on If = r and y & I, then (a) holds and ()5.13p gives the result. 
If now y I and = s, we are left with case (b) in (j5.12p : in particular yg = i^A(e-fc)- We only 
have to check that the assumption l^{ye-k) ^ ^''(j^A(e-fc) + [^^^]) implies the two inequalities in 
\5.9\ which is straightforward granted that, when e = 1, n is even (16. Sp . ■ 



Lemma 5.14. Let \k\ = s and y & I such that V{ye-k) G A''(z/A(e_A;)+ [^^] ). Define z = I+Z—Z 
with Z(e_fc) = —yZ\ and Z{et) = for t ^ —k. Then z belongs to P^n+i'^{A), contained 

in J n I, and V{zye^k) = 0. 

Proof. One checks easily that z belongs to / and V'{zye^k) = 0. It remains to show that Z 
belongs to arn+ii(A). We have, using notation 16.11 and Lemma [621 



Z G 0[^](A) ^ Zp^-'=^*^e_fc C A(t + [^]) for any t G Z 

^ Ze_fc G A'"(t - 4a_fe(t) + [^] ) for any t G Z 

^ r(ye_fc) G A^Xmaxjez{t-4a_fc(t)}+ [^]) 

^ r(ye_,)GA'-(z.A(e-fc)+[^]). 



With this in hand we are ready to conclude: 

Proposition 5.15. If the representation IndjA has a Whittaker model, there exists k G {— s,s} 
such that A contains the trivial character of Uk Ci J = Uk ri P{A^). 

Proof. Recall Proposition II. 6t if the representation Indj A has a Whittaker model, there exists 
k G {—2, —1, 1, 2} and y & I such that A contains the trivial character of yUky~^ H J. Note that if 
(fe, y) is such a pair, so is (/c, zyx) for any z € J (1 1 and x G / n NciUk)- 

Assume we are in this situation and pick such a pair {k,y). Since the restriction of A to L is a 
multiple of -0/3, Lemma EUD tells us that we must have |A;| = s and V{ye-k) G A"^ {i/A{e-k) + [^^])- 
The Proposition is then an immediate consequence of the following fact: 

(*) The double class (J H Ijy {I (1 NciUk)) contains an element of J Ci I, indeed of I (1 P{A^). 

Let us now prove this fact. Assume first that k = s = 2 (then r = 1). Using the standard Iwahori 
decomposition of ?/ G / and the fact that upper triangular matrices normalize U2 we may assume 
that y is a lower triangular unipotent matrix. Now since l^(ye_fc) belongs to A^(z^A(e-fc) + [^^4^]) 



25 



we may change y into zy where z is defined in Lemma 15.141 Note that z is also lower triangular 
unipotent, hence so is zy. Since \^{zye^k) = 0, zy has the following shape: 



A 













1 











b 


1 





V 








1/ 



The middle block 



zy 



centralizes Uo so the double class contains 



/ n Sp(y2) hence to P(A) n Sp(y2) 
lower triangular by upper triangular 

/ 

using conjugation hy w = 



/I 











which belongs to 






VJ p 











= P(A^), q.e.d. The case k = —s 

The cases k = s = 1 and k = —s - 
1 o\ 

or w~^, elements of GSp4(F) normalizing /, to get a 



-2 is identical, replacing 
-1 are obtained similarly, 



1 




convenient Iwahori decomposition for the initial element y. ■ 

We are at last ready to prove non-genericity of supercuspidal representations coming from case 
pvp . which will finish the proof of Theorem 12.11 The group C/^ fl J above is equal to J CiU 
for some unipotent subgroup U chosen as in Proposition 14.61 - indeed we have k G {— s,s} so Uk 
is a long root subgroup attached to the two-dimensional space in which we have a null stratum. 
We know from Proposition 14.61 that the restriction of A to J7fc fl J, a sum of non trivial characters, 
cannot contain the trivial character. Hence Indj A doesn't have a Whittaker model. 



6 Appendix: normalization of lattice sequences 

We gather in the first two subsections the technical information about direct sums of lattice se- 
quences that we need in several parts of the paper. Specifically, in most cases we deal with direct 
sums of self-dual lattice sequences in two-dimensional symplectic spaces and it will be convenient 
to homogeneize their F-periods and duality invariants. 

Then we proceed with preliminary results in view of the proof, in §6.4^ of Proposition 14.11 



6.1 Self-dual lattice sequences 

Let A be an OiT'-lattice sequence in a finite dimensional F- vector space V and let afe(A) be the 
corresponding filtration of A. We define: 

i^a{v) = max{i £Z;v £ A(i)} [v G V); fhig) = max{i £Z;ge ai(A)} {g e A). 

When V is equipped with a symplectic form h, lattice duality with respect to h is defined by 
L# = {v G V : h{v,L) C pp} for L an OiT'-lattice in V. An OiT'-lattice sequence A in F is self-dual 
if there is an integer d{A), the duality invariant of A, such that A(t)* = A(ci(A) — t). 

We now let A be a self-dual lattice sequence of period e in V and we pick a symplectic basis 
(e_2, e_i, ei, 62) adapted to A: there are non-decreasing functions : Z — > Z such that 

Mj)= Pp'^'^es (jGZ). (6.1) 

se{-2,-l,l,2} 

We will need the following straightforward property: 
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Lemma 6.2. Fork in {—2,-1,1,2}, we have 

(cfc) = -VA (e_fc) + d{K) - 1 = max {j - eak{j);j E Z}. 

Proof. Since the period of A is e, the two sets {j — eak{j);j G and {i € Z; afc(i) = 0} coincide: 
the valuation of is the maximum of either one. Next we use duahty to check that 

e_fc^A(x + (i) e_fc^A(-x)# <^ He^k^Pp"^'''^ ek) Pf ak{-x)<0. 



6.2 Normalization of our lattice sequences 

We start as in f|2] case OIip with an orthogonal decomposition V = ^ V'^ and not null skew 
simple strata [A*, n^, 0, in Endi?(y*). We let Ei = F[Pi]. We will find convenient to normalize 
the lattice sequences A* in such a way that their sum A is given by A(t) = A"^(t)_LA^(t) for any 
t E and is self-dual. This will be the case provided that A^ and A^ have the same period and 
d{A^) = d(A2) = 1 (see [20] for instance). 

The quadratic form v i— > h{v, f5iv) on has no non trivial isotropic vectors, since the anti-hermitian 
form 5i on defined by h{av,w) = tii:£;^/p{a6i{v,w)) for all a G Ei, v,w € V^, is anisotropic. Let 
Vi be a basis of over Ei. There is some Ui in Ei, satisfying Ui = —Ui, such that this form reads 
6i{xvi, yvi) = uxy for all x, y G i^j. Since Ei is tame over F, lattice duality is the same for h and 6i, 
namely A*(t)# = G ; 5i{v , {t)) C p^J. We have here {p^p^Vi)* = p%~*Vi with di = I - vpUi. 
Hence, up to a translation in indices, the unique self-dual 0£;.-lattice chain {Li)i<zi in satisfies 
one of three possibilities: 

(i) Ei is ramified over F and di = 0; 

(ii) Ei is unramified over F and = 0; 

(iii) Ei is unramified over F and di = \. 

In the two first cases we put L[{t) = and get a lattice sequence with duality invariant d'i = 1; 

in the third case we keep L'- = Li. In the ramified case we put Aj = L'-: it has period 4. We now 
need to put Aj = 2L^ in the second case, Aj = 4L- in the third case, and we get a normalization of 
our OEi- lattice sequences in Vj such that their period is 4 and duality invariant 1. We will use the 
following straightforward properties of Aj, in each of the three cases above: 

Lemma 6.3. Normalize the lattice sequence A* such that its period over F is 4 and (i(A') = 1. 

(i) IfEi is ramified over F then vk^{V' - {0}) = 2Z + 1 and VK^Pi) = 2uE,{(3i) + I; 

(ii) If Ei is unramified over F and Aj contains a self-dual lattice then v\.{V'^ — {0}) = 4Z + 2 and 
z.A,(A) = 2(2i/£;,(/3j) + l); 

(iii) If Ei is unramified over F and Aj does not contain a self-dual lattice then vp^^iy^ — {0}) = 4Z 
and i^Ai (A ) = 4:1^ Ei {Pi)- 

We need to relate, under these conventions, the valuations relative to the lattice chains Aj with the 
valuations over F of the quadratic forms h{v,f3iv), v £ V^. 
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Lemma 6.4. Let = e{Ei/F) and normalize the lattice sequence A' such that its period over F 
is 4 and d(A') = 1. For any v we have: 



Proof. For v in and a in Ei we have h{av,f3iav) = ^Ei/F{o)h{v, fiiv). The map v ^ h{v,(5iv) 
on Vi is thus constant on the sets kiit) — Aj(t + 1) hence factors through the valuation u^r. there 
is a map ^, defined on the image of and with values in Z, such that lyph^v, f3iv) = (f){i'js_^{v)) 
for any non zero v in V^. Periodicity over Ei implies (j){t + 2ej) = + Oj, t € Z: computing one 

value of (p is enough. We certainly have uphiy, (5iv) = — {^Eifii + ^EiSi{v,v)) whence: 

(i) if Ei is ramified over F, then Aj(l) = Aj(0) = Aj(l)'^ so I'Aiiv) = 1 implies UEiSi{v,v)) = 1 
and UFh{v,Piv) = ^{i^EiPi + 1); 

(ii) if Ei is unramified over F and Aj contains a self-dual lattice then Aj(2)^ = Aj(— 1) = Aj(2) 
so i^Aiiv) = 2 implies i/EiSi{v,v)) = 1 and VFh{v,j3iv) = i^EiPi + 1; 

(iii) If Ei is unramified over F and Aj does not contain a self-dual lattice then Aj(O)'^ = Aj(l) = 
tx7i?Ai(0) so i^Aiiv) = implies iyEiSi{v,v)) = and i'Fh{v,Piv) = VEif^i- 



6.3 Some intersections 

The following Lemma and Corollary were suggested by Vytautas Paskunas. In the proofs, we may 

(and often will) ignore the condition that the stratum and character be skew, since the results in 
the skew case follow immediately by restriction to G - that is, we actually prove the statements 
for c7-stable semisimple strata in GL4. To ease notation, this will be implicit so that, in the proofs 
below, U should be a cr-stable maximal unipotent subgroup of GL4, etc. 

First we need some notation in the semisimple case. For [A, n, 0, /3] a semisimple stratum in V with 
splitting V = 0^=1 V^, we write A = ^f^j^-^A^^ in block notation, where A^^ = Hom f{V^ , F') = 
V AV and V is the projection onto with kernel V^~^. For k = (ki,k2) G Z^, with ki>k2> 1, 
define 

and U''(A) = 1 + Ok. 

Lemma 6.5. Let [A, n,0,/3] he a semisimple stratum in A, with splitting V = , 1 < ^ < 2. 

Suppose also that F[j3] is of maximal degree over F, and let U he a maximal unipotent suhgroup of 
G. Write B for the centralizer in A of p. For k > m > 1, we have 

((u™(A) n s)u^(A)) nu = {\J^{A) n u). 

We remark that the proof below is just for the case which interests us here (so that there are at 
most 2 pieces in the splitting) but it is straightforward to generalize the Lemma to the case where 
there are an arbitrary number of pieces. 



28 



Proof. We will prove the corresponding additive statement. Writing U = 1 + N, bm = cim(A) CiB, 
it is: 

(bm + Ofe) n N = Ofc n N. 

Notice that, since B = F[f3], the lattice bm contains no non-trivial nilpotent elements. We will 
show that, for 1 < m < k, 

(bm + Ofe) n N C (b„+i + Ofc) n N 

and the result follows at once by an easy induction. So suppose e G b^ is such that (e + Ofc) nN 7^ 0. 
In particular, (e + ttm+i) fl N 7^ so there exists s > such that G cism+i- But then G bsm+i 
so (by [3], working block- by-block) , the coset e + b^+i contains a nilpotent element, which must 
be 0. We deduce that e G bm+i, as required. ■ 

For [A, n, 0, (3] a semisimple stratum in V with splitting V = 0^=^ V^, we write Mgp = Auti;'(y^) x 
AntF{V^), Usp = l + A^^, Psp = MspUsp and Usp = 1 + A^K 

Corollary 6.6. Let [A, n, 0, (3] be a semisimple stratum in A, with splitting V = ©'=1 V^, 1 < I < 2, 
and let U he a maximal unipotent subgroup of G. Write B for the centralizer in A of (3. We suppose 
that 

(i) U has an Iwahori decomposition with respect to {Mgp, Psp); 

(ii) U n B^ is a maximal unipotent subgroup of B^ . 

Then, for ki > ■ ■ ■ > ki > m > 1 and k = {ki, . . . , ki), we have 

((U'"(A) n B)\]^{K)^ nu = (u™(A) nBn u){v^{A) n u). (6.7) 

Again the statement is easily generalized to the case when the splitting has more than two pieces. 
Note also that, in the simple case, condition (i) is empty while condition (ii) is implied, for example, 
by the condition '0/3|(7der = 1 (0 Proposition 2.2). This is not true for semisimple strata. 

Proof. First we reduce to the simple case. We notice that U''(A) and B^ C Msp have Iwahori 
decompositions with respect to {Msp,Psp)- Since U also has such a decomposition, we reduce to 
proving that we have equality in (j6.7p when we intersect both sides with Usp, with Mgp and with 
Usp respectively. Since B C Mgp, this is immediate for the unipotent radicals Usp, Usp- Hence we 
are reduced to the intersection with Mgp, which, block-by-block, is just the simple case. 

So now suppose [A, n, 0, /3] is a simple stratum. As in Lemma l6.5( we will prove the corresponding 
additive statement: writing ?7 = 1 + N, it is: 

(b^m + ak)nN= (b^nN) + (oknN), (6.8) 

where k = ki. We will reduce to the case where E = F[I3] is maximal and invoke Lemma 16.51 
Write d= \E : F\; then, in the flag corresponding to U , 

{0} = Fo C C • • • C Fat = y, 

the subspace Vdi is an i?-subspace, for < i < N/d. Let C/q = 1 + No be the unipotent subgroup 
corresponding to the maximal E'-flag 

{{}} = V^ CVdCl ■ ■ ■ cVdi CI ■ ■ ■ cVn = V 
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and let Pq = 1 + Pq the corresponding parabolic subgroup. There exists an E'-decomposition 
V = e^J^Wi of V such that for < i < N/d, Vdi = ®i=iWj and such that for every t G Z, 

A(t) = ©^^f A(t) n Wi (as a suitable variant of, e.g., [22] §11.1). Let Lq be the corresponding Levi 
component of Pq and let C/q = 1 + No be the unipotent subgroup opposite Uq with respect to Lq. 

The lattices hm and have (additive) Iwahori decompositions with respect to No,Po ([4J §10) so 
{bm + Ofe) n Po = n Lo + b„ n No + ttfc n Lo + ttfc n No 

and we have 

(b„, + Ofc) n N = {bra + afc) n Po n N = (b^ n Lo + ak n Lo) n N + hm n No + Ofc n No 

Hence we are reduced to showing 

(bm n Lo + afc n Lo) n N = bm n Lo n N + afc n Lo n N, 

which is the same as (16. Sh in Lo, where we can work block- by-block. In each block, the field 
extension E is maximal, so we have indeed reduced to the maximal case, which is Lemma 16.51 ■ 



Finally, we will need one more similar result, in the case of a minimal semisimple stratum - in this 
case the extra conditions of the previous Corollary are not satisfied. 

Lemma 6.9. Let [A, n, 0, /?] he a skew semisimple stratum, with splitting V = V'^ @ V'^ , 1 < I <2. 
Suppose that Pi = 1^/31* is minimal, for i = 1,2 and that [A,n,n — 1,(3] is not equivalent to a 
simple stratum. Assume also (without loss of generality) that ni > n2 > 1. Put ki = \^^] + 1, 
for i = 1,2, and k = {ki,k2). Then = (U^(A) fi -B)U''(A) and, if U is a maximal unipotent 
subgroup of G on which tpp defines a character: 

H^nu = \J^{A) n u. 

Proof. We work under the conventions made in the previous subsection. The subgroup U is 
necessarily given (see Proposition 13. 4p by a flag of the following form: 

{0} C {vi + V2) C {vi + V2,v-i - w_2) C {vi,V2,v^i - V-2) C y, 

where Vi £ are non zero vectors such that h{vi,[5ivi) = —h{v2, f32V2) = n and V-i = n~^j3ivi, 
w_2 = —lJ'~^hv2- In particular {vi,V-i} is a symplectic basis for V^, adapted to A*. 

That = (U^(A) n B)\]^{h) is clear from the definition. For the intersection property, we will 
need: 

Lemma 6.10. Put v = i'Ai(^^i) — '^A2(^2) cLn-d let Ei_j be the linear map sending vj to Vi and all 
other basis elements to 0. We have 

i^h{Ei,2) = 1^, yA{E^2-i) =ni-n2-v. (6.11) 

Furthermore, if ni > n2, then < < ni — n2. 

Proof. Computing the valuations of Ei^2 and E-2,-1 is simple checking. Lemma 16.41 combined 

with our asssumption on vi, V2 implies u = _| E2P2 ^ rpj^^ inequality < u < ni — n2 

ei 62 

then follows from Lemma 16.31 ■ 
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Write [/ = 1 + N. Then the elements of N can be written (as matrices with respect to the basis 
{?;i,?;_i,i;2,W-2}) 

/a e —a c ^ 
-d -b d -b 
a e + / —a c + f 
\d b -d b / 

for a, b, c, d,e, f € F. 

Now suppose that x as above also Hes in the lattice (in block matrix form, each block 2x2) 

dm Clfci 



for some m < ki. Then, using (j6.1ip . we get: 

• aE2,i e Ofc^ so aEi^i = a£'i,2-E'2,i € 0^-^+,^; 

• bE^i^^2 £ cifci so 6£'_i^_i = 5£^_i^_2-E'-2,-i € afc^+„^_„2_,y; 

• (C + /)^2 -2 G flfe so (C + _2 = (C + f)Ei^2E2-2 G afca+vJ 
Ci?l -2 G Clfel so -2 G ainin{A:i,fc2+,/} 

and /-El -1 = /-Bi _2i?-2,-l G amin{fei+ni-n2-i/,fc2+rii-n2}- 

(e + f)E2-i G Ofci so (e + _i = (e + /)^i,2^2 ~i G 0^,+,,. 

Writing t = min {z^, ni— n2— z/, /c2+?T'1— ?^2 — ^i} > 0, we have aEi^i, ei?i^_i G o^^+i C a-m+i. 

In particular, looking at the "top-left" block of x, we have 

a e\ / 0\ 
Now we prove that, for m <k2 and k = (A;i, A;2), 

((U'"(A) n s)u'^(A)) n f/ = u''(A) n u. 

By Lemma |6.5[ we have 

((u™(A) n s)u'^(A)) n [/ c ((U'"(A) n 5)u'=^ (A)) r\U = \]^^ (A) n c/. 

Then we need only prove, for k2 < m < ki (the additive statement) 

But we have seen above that, if e G ^ is such that e + Ok contains an element of N, 

then e + ^'^"^+^ ^ also contains a nilpotent element. But then (as in the proof of Lemma l5.5p 
e + f H j also contains a nilpotent element, which must be 0. Hence e S bm+i, as required. 
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6.4 Proof of Proposition 14.11 



It remains to prove Proposition 14.11 If ^ € C(A, 0, (3) is a skew semisimple character and C/ is a 
maximal unipotent subgroup of G such that ip/slu^er ~ then 

As in the previous section, we actually prove the same statement for u-stable semisimple strata in 
GL4, and the result follows by restriction to G. We remark that a skew element /? generating a 
field such that [F[P] : F] = 2 is necessarily minimal. We proceed on a case-by-case basis: 

Simple case (Cases (I) and (II)) We proceed by induction on r = —ko{(3,A). We make the 
following additional hypothesis: 

(*) there exists a simple stratum [A, n, r, 7] equivalent to [A, n,r, /3] such that ip'y\ud^r ~ ^■ 

In particular, we can then use the inductive hypothesis for the stratum [A, n, 0, 7] with the same 
unipotent subgroup U. Note that this hypothesis is certainly satisfied when P is minimal, since we 
can take 7 = 0. We will show later that it is also satisfied in the other cases that are relevant to us 
here. 

We have H^{(3,K) = (Ui(A) nB)i?[3]+i(/3, A) so, by Corollary EJl H^r\U = i/lil+inC/. Moreover, 
F[5]+n/?,A) = if[5]+i(7,A) and ^l^iji+i = 0o|j:^[5]+iV'/3-7, for some 6*0 G C(A,0,7). But, by the 
inductive hypothesis, 6q agrees with ■0^ on ^^[2]+^ n U and the result follows. 

To finish, we must show that (*) is satisfied. The only case to consider is when [A, n, 0, [3] is a 
skew simple stratum with F[f3] maximal (of degree 4) and r = —kQ{P,A) < n. Then [A, n, r, /?] is 
equivalent to some skew simple stratum [A, n, r, 70], with 70 minimal and F[^q\ of degree 2 over F. 
Note that, since p 7^ 2, all extensions are tame here. Note also that the flag corresponding to the 
unipotent subgroup U is given by = {v,(3v, ■ ■ ■ ,f3'^~^v), for some v £V with h{v,(5v) = 0. Also 
put C = /i(/3f , (j'^v) / 0, and d = -vf{C)- 

Let P{X) = X'^ + A G F[X] be the minimal polynomial of 70 and put 6-2 = 1^, = Pv, 
ei = C-^P{P)v, 62 = -PC~^P{P)v - k(3v, where k = C~'^h{[5v, 0^v) - 2AC~^ (this is a symplectic 
basis). With respect to this basis, (3 has matrix 





-A 







1 





k 








c 





A 


Vo 





-1 





for some /U € -F. For i,j E {ibl,ib2}, write E'jj for the linear map sending Cj to and all other 
basis vectors to 0. We then have 

Write /3 = 7o + Co, with cq G a_r- Then + A = co7o + 7oCo + Cq G a-n-r- From the matrix 
description of /3, we get that 








-(/u + /cA) 


\ 





K 





(^ + k\) 


C 





kc 







-c 





/ 



Then: 
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• (/i + kX)E^i^2 G a-n-r and (/^ + kX)E_2,2 = (/^ + A;A)£;_2 -i-E_i,2 G a_r-; 



Hence 



c = 



/O n + kX\ 



C 

\0 / 



G a-r- 



Put 7 = /? — c. Then [A, n,r, 7] is a stratum as required by hypothesis (*). 

Minimal semisimple case (Case (III)) Now suppose [A, n, 0, f3] is a skew semisimple stratum, with 
spUtting V = V^^V'^, 1 <l <2, that Pi = 1^/31' is minimal, for i = 1, 2 and that [A, n,n-l, /3] 
is not equivalent to a simple stratum. Assume also (without loss of generality) that rai > 722 > 1. 
Put ki = + for i = 1,2, and k = (/ci,/c2). Then, by Lemma = (U'^{A)nB)\J^{A) and 
//^ nU = U''(A) n U. But, by definition of 9, it agrees with ipp on U''(A) so the result follows. 

Degenerate semisimple case (case (IV)) Now suppose [A, n, 0,/3] is again a skew semisimple 
stratum, but with P2 = 0. In this case it is straightforward to see that the flag defining U must be 
given by 



where V2 G V"^ and vi £ V^. In particular, U satisfies the extra conditions in Corollary 16.61 Putting 
ki = [t^] + l and /C2 = 1, we have = (U^A) n B)\J^{A) so, by Corollary EHl 



As in semisimple case (i), the result now follows by the definition of semisimple characters. 

Non-minimal semisimple case (case (III)) Finally, suppose [A, n,0, /3] is a skew semisimple 
stratum, with splitting V = ^ V'^, 1 < / < 2, that (3i = l^f3V is minimal, for i = 1,2 but that 
[A, n,n — 1, (3] is equivalent to a simple stratum. Let r = — A;o(/3, A). As in the simple case, we will 
show that 

(*) there exists a simple stratum [A,n,r, 7] equivalent to [A,n,r,(3] such that tp'y\uder ~ ^■ 

(Indeed, 7 will be minimal.) The proof is then the same as that in the simple case, since we can 
use the simple case for 7. In this case we invoke Lemma 16.51 to show that nU = H^2^~^^ n U. 

As in the simple case, the flag corresponding to the unipotent subgroup U is given by = 
{v,Pv,--- ,f3'^~^v), for some v £ V with h{v,f3v) =0. So v = vi + V2, with Vi G such that 
h{vi, (3iVi) = —h{v2, (32^2)- Note that, for i = 1,2, {vi, PiVi} is a basis for V^, with respect to which 
Pi has matrix 



for some Aj G F. Also, let Ei denote the linear map in A** which send PiVi to Vi and Vi to 0; then 
Ei G ajf. 

Let [A,n, r, 7o] be a skew simple stratum equivalent to [A,n,r,P], with 70 G Msp, and let X'^ — A 
be the minimal polynomial of 70 over F. For i = 1,2, let ji E j4** have matrix 



{0} cVi = {V2) CV2 = {V2,vi) cV3 = Vi(BV^ cVi = V, 



H^nu = (u^(A) n n u){\]^{A) nu) = \]^{A) n u. 
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with respect to the basis {vi,(3iVi} of F*. Since /3 — 70 G a_r, we get that A — Aj G ci-n-r' 
2 = 1,2, so (A — Xi)Ei G a!!^. Hence Pi — G a!!^ and 7 = 71 + 72 is as required, since = j3v. 

This completes the proof of Proposition 14.11 ■ 

Remarks It surely will not have escaped the reader's notice that the methods in each case are 
rather similar. It may well be possible to unify the cases into a single proof but we have not been 
able to do this. We also note that we could not have used |5] Lemma 2.10 here, since the proof 
given there unfortunately does not work. It seems likely that the result there is true (at least in 
the tame case, as here) but we have not (yet) been able to find a proof. 
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